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(54) FACE IMAGE MODEL RETRIEVAL PROCESSING METHOD 

(57)Abstract: 

PURPOSE: To provide a simple face image model 
retrieval processing method with which the features of a 
face area can be extracted with high accuracy at a high 
speed from an object as efficiently described data. 
CONSTITUTION: In a step by a face area mosaic feature 
extracting part 40, a gray level image including the face 
area is made into mosaic and while utilizing the fact that 
the pattern of this mosaic is made different inside and 
outside the face area, the face area is extracted as face 
area mosaic feature data. Next, in a step by a face image 
model retrieving part 60, the class name of model 
classification is retrieved while using those face area 
mosaic feature data. Thus, in the case of extracting the 
face area, attention is not paid on the edge part of the 
object but paid on the density pattern of the object area 
so that not only the high-accuracy and high-speed 
extraction/retrieval of the face area but also the 
classification of models for retrieval can be performed. 









-.■■" : MV. : 






«<f MNb> 


xn m 






u a ' 






.'• I* . 







m 

r t 



LEGAL STATUS 

[Date of request for examination] 1 2.1 1.1 999 

[Date of sending the examiners decision of 03.12.2002 
rejection] 

[Kind of final disposal of application other than 
the examiner s decision of rejection or 
application converted registration] 

[Date of final disposal for application] 

[Patent number] 

[Date of registration] 

[Number of appeal against examiners decision 
of rejection] 



http://www19.ipdl.jpo.go.jP/PA1/result/detail/main/wAAAzta4UVDA408221547P1.h... 03/10/20 



Searching PAJ 



2/2 ^— v 



[Date of requesting appeal against examiner s 
decision of rejection] 

[Date of extinction of right] 

Copyright (C); 1998,2003 Japan Patent Office 



http://www19jpdljpo.gojp/PA1/result/detail/main/wAAAzta4UVDA408221547P1.h... 03/10/20 



[JP,08-221547,A] 
* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

l.This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2 **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim l] The face picture type reference art characterized by to have the face field mosaic 
feature-extraction step which mosaic-izes a shade picture including a person face field, and extracts a face 
portion as face field mosaic feature data, and the face picture type reference step which look for the 
nearest feature class data in the mosaic feature class set given beforehand about the aforementioned face 
field mosaic feature data, and output the class name as a reference result. 

[Claim 2] The face picture type reference art according to claim 1 characterized by giving the mosaic 
feature class set beforehand about two or more person face pictures specified beforehand by the step 
extracted as each face field mosaic feature data, and the step which carries out the class division of the 
face field, mosaic feature data aggregate of two or more person face pictures by which extraction was 
carried out [ aforementioned ] based on the degree of similar, and is considered as the mosaic feature class 
set. 

[Claim 3] The face picture type reference art according to claim 1 or 2 characterized by providing the 
following. A face field mosaic feature -extraction step is. a head rough search step which searches the 
position where a head is rough using the head rough search mosaic dictionary which was able to give 
beforehand the person face image data mosaicized coarsely, and obtains near [ a head center 1 positional 
information. The particular-part detailed search step which mosaic-izes finely the particular part on the 
aforementioned person face image data based on the aforementioned near [ a head center ] positional 
information, searches the detailed position of the aforementioned particular part using : the 
particular-part detailed search mosaic dictionary which was able to give the obtained specific field mosaic 
feature data beforehand, and obtains face center detailed positional information The face field 
mosaic ized step which asks for the position of the face field on the aforementioned face image data 
correctly based on the aforementioned face center detailed positional information, mosaic-izes the face 
field on the aforementioned face image data using the obtained face field positional information, and is 
outputted as face field mosaic feature data 

[Claim 4] The face picture type reference art according to claim 3 characterized by the particular part on 
face image data being the eye section. 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] this invention relates to the face picture type reference art for knowing whether 
it is close to which face picture class whose face image data which carries out the type classification of the 
face image data of a large number obtained by the camera input of an object face or the scanner input of a 
photograph of his face beforehand, and which was both newly inputted has been classified. 
[0002] 

[Description of the Prior Art] Although the face field on a picture needed to be detected first to carry out 
the type classification of the face picture (shade picture), and search, while the edge detection of the 
picture was carried out by differential operation etc. and thinning or ridge line tracking extracted the 
obtained edge field as a diagram, in the conventional type reference art, it was common by analyzing the 
geometrical structure of this diagram to have carried out a type classification and to have searched. 
[0003] 

[Problem(s) to be Solved by the Invention] However, in the type reference art of the above-mentioned 
conventional picture, since the loose concentration change field and shadow field of an inclination are 
included in a picture in many cases, it is almost next to impossible to detect an edge portion correctly. And 
since processing of differential etc. is included, the time which computer-processing requires cannot be 
disregarded. 

[0004] Therefore, the present condition is that the face picture type reference art in practical use level is 
not announced until now. 

[0005] The technical problem, which aims at solution by this invention is proposing a simple face picture 
type reference art based on this further while solving the difficulty of the conventional technology in 
which it tends to be based on detection and analysis of an edge portion, and extracts and judges objects 
(face field etc.). 
[0006] . 

[Means for Solving the Problem] The face field mosaic feature-extraction step which mosaic-izes a shade 
picture including a person face field, and extracts a face portion as face field mosaic feature data in order 
to solve the above-mentioned technical problem, It looks for the nearest feature class data in the mosaic 
feature class set given beforehand about the aforementioned face field mosaic feature data, and let the 
face picture type reference art of composition of having the face picture type reference step which outputs 
the class name as a reference result be a means. 

[0007] In the above-mentioned face picture type reference art, it is suitable to consider about two or more 
person face pictures specified beforehand as the composition which gives the mosaic feature class set 
beforehand by the step which extracts as each face field mosaic feature data, and the step which carries 
out the class division of the whole face field mosaic feature data aggregate of two or more person face 
pictures by which extraction was carried out [ aforementioned ] based on the degree of similar, and 
consider as the mosaic feature class set. 

[0008] In the above-mentioned face picture type reference art moreover, a face field mosaic 
feature-extraction step The head rough search step which searches the position where a head is rough 
using the head rough search mosaic dictionary which was able to give beforehand the person face image 
data mosaic-ized coarsely, and obtains near [ a head center ] positional information, Based on the 
aforementioned near [ a head center ] positional information, the particular part on the aforementioned 
person face image data is mosaic-ized finely. The particular-part detailed search step which searches the 
detailed position of the aforementioned particular part using the particular -part detailed search mosaic 
dictionary which was able to give the obtained specific field mosaic feature data beforehand, and obtains 
face center detailed positional information, Based on the aforementioned face center detailed positional 
information, it asks for the position of the face field on the aforementioned face image data correctly. 
Considering as the composition which has the face field mosaic-ized step which mosaic-izes the face field 



on the aforementioned face image data using the obtained face field positional information, and is 
outputted as face field mosaic feature data It is suitable to consider as the composition whose particular 
part on the suitable further above-mentioned face image data is the eye section. 
[0009] 

[Function] In the face picture type reference art of this invention, a shade picture including a face field is 
mosaic-ized, a face field is extracted as mosaic feature data using these mosaic patterns differing in the 
interior and the exterior of a face field, and a class division and reference of a type classification are 
performed using this. Thus, without paying attention to the edge portion of an object, extraction, a class 
division, and reference of a highly precise and high-speed face field are enabled by paying one's attention 
to the concentration pattern of an object field in extraction of the face field. 
[0010] 

[Example] Hereafter, a drawing explains the example of this invention in detail. 

[0011] Drawing 1 is drawing showing the example of the portion about the calculation / acquisition 
method of the mosaic feature class data set [ in / drawing 1 / for the example of the portion about the face 
picture type reference art of this invention / in drawing 2 ]. 

[0012] In drawing 1 and drawing 2 , 10 is the object face or photograph of his face set as the object of a 
picture input. 

[0013] 20 is the image data input section which bears the step which inputs an object face or a photograph 
of his face with a camera, a scanner, etc., and stores the obtained image data in the picture data buffer 30. 
[0014] 40 mosaic-izes the image data which is incorporated by the image data input section 20 and stored 
in the picture data buffer 30, detects a field which is in agreement by the mosaic pattern to the typical 
face field beforehand given by scanning the mosaic data top, and a certain within the limits, and bears the 
step which outputs the mosaic data of the portion as face field mosaic feature data 50 to an input picture. 
[0015] The face picture type reference section 60 of drawing 1 searches the class which investigates the 
degree of coincidence with each mosaic feature class on the mosaic feature class data set 70, and is best in 
agreement about the face field mosaic feature data (mosaic pattern) 50 obtained in the face field mosaic 
feature-extraction section 40, and bears the step which outputs the class name as a reference result 80. 
[0016] By repeating and applying the image data input section 20 of drawing 2 , and the face field mosaic 
feature-extraction section 40, the mosaic feature data collection section 90 of drawing 2 collects the face 
field mosaic feature data 50 to face image data different, respectively one by one, and bears the step 
which collects these and obtains the mosaic feature data set 100. 

[0017] The face picture clustering section 110 carries out the class division of the mosaic feature data set 

100 for example, by the multidimensional- scale method etc., and bears the step which outputs a set of 

representation [ the mosaic feature data belonging to each class ] (for example, average) as a mosaic 

feature class 70. In addition, database-izing is suitable for this mosaic feature class 70. 

[0018] Drawing 3 is the block diagram showing the example of drawing 1 and the face field mosaic 

feature-extraction section 40 of drawing 2 . The content is explained below about this drawing. 

[0019] All the field mosaic-ized sections 401 bear the step which mosaic-izes by the coarse block size 

which has crossed the image data on the picture data buffer 30 to all fields, and outputs all fields as 

mosaic feature data 402. 

[0020] The head rough search section 403 bears the step which outputs the midpoint of both the near eye 
position as near [ a head center ] positional information 405 while it scans the picture data buffer 30 
coarsely and detects the head field on a picture by using the mosaic feature (for example , 4 (width )x5 
Gength ) block ) about the head (portion above a neck ) of a typical general configuration as a head rough 
search mosaic dictionary 404 . 

[0021] The eye section mosaic-ized section 406 bears the step which mosaic-izes the near field which 
contains both the eyes given by the near [ a head center ] positional information 405 about the image data 
of the picture data buffer 30 by the finer block size, and outputs it as eye section mosaic feature data 407. 
[0022] The **** detailed search section 40J8 About the field containing both the eyes of a typical general 
configuration, for example, by using the 8(width) x3 (length) block mosaic feature as a **** detailed 
search mosaic dictionary 409 (a block size quite smaller than that of the head rough search mosaic 
dictionary 404 being taken (about [ for example, ] 1/5)) While scanning the eye field mosaic feature data 
407 in a detail more and detecting a **** field in a detail more, the step which outputs the midpoint of 
both the obtained eyes as face center detailed positional information 410 is borne. 

[0023] While a face center and the face field spotting section 411 determine the position of both the eye 
field correctly using pixel value information, such as concentration change and a histogram, [ near the 
position given by the face center detailed positional information 410 ] on the picture data buffer 30 The 
step outputted as face field positional information 412 is borne by determining the regio oralis similarly 
and determining a face field (field determined between a face width, and a frame and a jaw) by both 



positional information etc. 

[0024] The field given by the face field positional information 412 about the picture data buffer 30 is 
mosaic-ized, for example, horizontal x length acquires the mosaic feature (generally a block size is a grade 
a little smaller than that of the eye field mosaic feature data 407) of 16x16 and 10x3 (it corresponds to the 
portion of a jaw), and the face field mosaic-ized section 413 bears the step which outputs this as face field 
mosaic feature data 50. 

[0025] In addition, when the size of the face picture inputted from the image data input section 20 is not 
fixed, it is suitable to change the size of a mosaic and to look for a mosaic dictionary 

[0026] In the face picture type reference art which consists of steps of each part by the above example A 
shade picture including a face field is mosaic-ized in the step by the face field mosaic feature-extraction 
section 40. In the step extract a face field as face field mosaic feature data using these mosaic patterns 
differing in the interior and the exterior of a face field, and according to the face picture type reference 
section 60 In the step according to the mosaic feature data collection section 90 in searching the class 
name of a type classification **** using the face field mosaic feature data By collecting the 
above-mentioned mosaic feature data about many face pictures, and performing a class division of a type 
classification in the step by the face picture clustering section 110 That is, solution of the difficulty of the 
conventional technology which enables extraction, a class division, and reference of a highly precise and 
high-speed face field, and makes a subject the edge detection and shape analysis of a field is aimed at in 
extraction of a face field by paying one's attention to the concentration pattern of an object field without 
paying, attention to the edge portion of an object. Moreover, it becomes possible by using the mosaic 
feature data of a face field to perform the feature description of an input face picture efficiently. 
[0027] At the face field mosaic feature-extraction step by the step of each part shown in drawing 3 in the 
face picture type reference art of the above-mentioned example In the step by the head rough search 
section 403 in extracting the face field mosaic feature data, and the step by the ,**** detailed search 
section 408 the mosaic feature of various sizes -- being gradual (all the field mosaic features, the eye field 
mosaic feature) - since it uses - the position of the face field on a picture - easy -- search and 
determination -- it can do -- coming - more - much more -- extraction precision - improving - the 
processing time - it can be shortened . 

[0028] In addition, in the detailed search for asking for the above-mentioned face center position detailed 
information, although it is also possible to mosaicize the particular part on face pictures other than .****, 
and to search for it, by mosaic izing the eye section in which a concentration pattern appears 
characteristic most, and searching for it, the search decision of a face field can be made still easier, and 
extraction precision can be raised further further. 

[0029] Moreover, in operation of this invention, since there are little the processing time and memory 
space which are required of a computer, a simple face picture type search method with a personal 
computer etc. can be constituted. 
[0030] 

[Effect of the Invention] As explained above, according to the face picture type reference art of this 
invention, a shade picture including a face picture is mosaic-ized. Since a face field is extracted as mosaic 
feature data and it was made to perform a class division and reference of a type classification by using 
that these mosaic patterns differ in the interior and the exterior of a face field using this it becomes 
possible to perform efficiently the feature description which is an input face picture by attaining 
extraction, a class division, and reference of a highly precise and high-speed face field, and being able to 
aim at solution of the difficulty of the conventional technology which makes a subject the edge detection 
and shape analysis of a field, and using the mosaic feature of a face field 

[0031] Moreover, in the above, when a face field is rough -searched first and the detailed search of the 
particular part of a face field is next made to be carried out in having obtained face field positional 
information and extracting the face field mosaic feature data, using the mosaic feature of various sizes 
gradually, while being able to raise extraction precision further, the processing time can be shortened 
further. 

[0032] Moreover, when the eye section is adopted as a particular part of the face field in the 
above-mentioned detailed search, search can become still easier and extraction precision can be raised 
further. 

[0033] Furthermore, in operation of this invention, since there are little the processing time and memory 
space which are required of a computer, a simple face picture type search method with a personal 
computer etc. can be constituted. 



* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated: 
3.1n the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing ll The block diagram showing the example of the portion about the face picture type reference 
art of this invention 

TDrawing 21 The block diagram showing the example of the portion about the acquisition method of the 
mosaic feature class data set in drawing 1 

fDrawing 31 The block diagram showing the example of the portion which takes charge of the face field 

mosaic feature -extraction step in drawing 1 or drawing 2 

[Description of Notations] 

10 Things or photograph of his face 

20 Image data input section 

30 -■ Picture data buffer 

40 - Face field mosaic feature-extraction section 
60 - Face picture type reference section 
70 -- The mosaic feature class data set 
110 -- Face picture clustering section 
401 All the field mosaic-ized sections 

403 Head rough search section 

404 -- Head rough search mosaic dictionary 
406 - Eye section mosaic-ized section 

408 - section detailed search section 

409 - section detailed search mosaic dictionary 

411 -- A face center and the face field spotting section 
413 Face field mosaic-ized section 



[Translation done.] 



